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From column 4, line 4 5 to column 9, line 10: 
[0009] 

[Preferred Embodiment of the Present invention] 

Referring now to FIG. land FIG. 2, one preferred embodiment 
of the present invention will be described hereinbelow. FIG. 
1 is a block diagram of a remote control system of the present 
embodiment, and FIG. 2 is a brief sketch of the remote control 
system of the present embodiment . 
[0010] 

■Referring now to FIG. 2, construction machine 10, such 
as a power shovel and a bulldozer, is located in construction 
site A, such as a disaster-stricken area, which is under 
restoration works. Construction machine 10 is equipped with 
TV camera 11 for recording the operation of construction machine 
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10, radio communication device (second radio communication 
means) 12, and others. As shown in FIG. 2, at center B away 
from site A, there are provided operator console 20, monitor 
21 for showing video images taken by TV camera 11, personal 
computer (hereinafter called PC) 22 for use in remote control 
(hereinafter called PC) , and radio communication device (first 
radio communication means) 23 for receiving/transmitting 
signals from/to radio communication device 12 of construction 
machine 10. On operator console 20, there are mounted backup 
remote control unit 201 (e.g., a joystick) which is used to 
manually operate construction machine 10 from a remote place, 
and backup remote control unit 202 (e.g. , a joystick) which is 
used to manually operate TV camera 11 to rotate and zoom in/out. 
Operator 24 who stays at center B controls these remote control 
units, 201 and 202. Operator 24 wears microphone (speech input 
means) 25 on his head, through which microphone 25 operator 24 
inputs his speech instructions for controlling construction 
machine 10 and TV camera 11. 
[0011] 

Referring now to FIG. 1, a remote controlling system of 
the present embodiment will be described hereinbelow. Both 
microphone ( speech inputting means ) 25, through which operator 
24 ' s speech instructions for controlling TV camera 11 are input , 
and construction machine 10 are coupled to PC 22 . Further, radio 
communication device 23 is coupled to PC 22 via communication 
interface 26, and remote control units 201, 202 mounted on 
operator console 20 are also coupled to PC 22. Still further. 
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monitor 21 is coupled to PC 22 via communication interface 26 
and display control means 27, which takes the image data 
transmitted from communication interface 26 via PC 22, and then 
converts the image data into graphic display signals to output 
to monitor 21. PC 22 has speech instruction recognizing means 
221 , which recognizes a speech instruction by analyzing the voice 
wave signal output from microphone 25, and operation command 
converting means 222, which converts the speech instruction 
recognition result obtained by speech instruction recognizing 
means 221 into an operation command to construction machine 10 
and TV camera 11. The operation command output from operation 
command converting means 222 is transmitted from radio 
communication device 23 to construction machine 10 via 
communication interface 26. 
[0012] 

Speech instruction recognizing means 221, as shown in FIG. 
1 , has speech-analyzing/characteristics-extractingmeans 221A, 
storage means 221B, comparing means 221C, and operation 
instruction selecting means 221D. Speech -analyzing/ charac- 
teristics-extracting means 221A analyzes a voice wave signal 
received from microphone 25 to extract the characteristics of 
operation instructions and a parameter characteristic of the 
vocalization of operator 24. Storage means 221B stores time 
series of such characteristic parameters , which has been obtained 
by speech-analyzing/characteristics -extracting means 221A, as 
standard patterns . Comparing means 22 IC compares a wave of voice 
produced by operator 24 , whose standard patterns have been stored 
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in storage means 221B, with the standard patterns stored in 
storage means 221B to calculate the degree of similarity between 
them. Operation instruction selecting means 221D outputs, as 
its recognition result, an operation instruction that 
corresponds to the standard pattern which revealed the highest 
degree of similarity in the above calculation by comparing means 
221C. 
[0013] 

In FIG. 1, radio communication device 12 equipped to 
construction machine 10 receives an operation command from radio 
communication device 23 of center B, and also transmits video 
signals taken by TV camera 11 to center B. Radio communication 
device 12 is connected with control circuit (control means) 14 
and image processing means 15 via communication interface 13. 
Image processing means 15 processes the video images recorded 
by TV camera 11 into the image data for transmission before the 
image data is output to radio communication device 12 . Control 
circuit 14 controls construction machine 10 and TV camera 11 
to obey an operation command demodulated by radio communication 
device 12. Drive unit 101 of construction machine 10 includes 
hydraulic circuitry that drives construct ion machine 10 and makes 
it exert its functions . This hydraulic circuit is under control 
of control circuit 14, which operates according to operation 
commands, so that operator 24 can remotely control construction 
machine 10 with speech instructions . Drive unit 111 of TV camera 
11 includes a motor for making TV camera 11 zoom in and out and 
a motor for moving TV camera 11 vertically and horizontally. 
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and these motors are controlled by control circuit 14 which 
operates according to operation conunands, so that operator 24 
can remotely control TV camera 11 with speech instructions. 
[0014] 

An operation of the system of the present embodiment will 
be described hereinbelow. The remote control system of FIG. 
1 is activated, and speech instruction recognizing means 221 
is made into a mode for registering standard patterns . Under 
this state, operator 24 vocalizes a phrase that is expected to 
be required for operating construction machine 10, for exeunple, 
instructions such as "go right," "go left," "go forward," and 
"dig," and such vocalization is taken by microphone 25 to be 
converted into a voice signal. Speech-analyzing/charac- 
teristics-extracting means 221A takes and analyzes the voice 
signal output frommicrophone 25, and extracts the characteristic 
of the operation instruction and a parameter characteristic of 
the vocalization of operator 24. A time series of the 
characteristic parameter is registered in storage means 221B 
as a standard pattern, and the pattern is associated with its 
character representation (spelling, and so on). The forgoing 
sequence of processes is repeated for all the phrases that are 
expected to be required for operating construction machine 10, 
and time - series standard patterns are registered in storage means 
221B. In the similar manner, operator 24 vocalizes the phrases 
that are expected to be required for operating TV camera 11, 
for example, instructions such as "zoom in," "zoom out," "turn 
rightward," "turn leftward," and time series of the 
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Characteristic parameters obtained from 

speech-analyzing/characteristics -extracting means 221A are 
registered in storage means 221B as standard patterns. 
[0015] 

Next, the remote control system of FIG. 1 is made into 
a remote controlling mode, video images, taken by TV camera 11, 
of the operations of construction machine 10 are monitored in 
real time on monitor 21 in center B. At that time, when remotely 
controlling construction machine 10 with speech instructions, 
operator 24, whose standard patterns are registered, vocalizes 
an operation instruction , for example , "go right , " while viewing 
the monitor screen of monitor 21. The vocalization is picked 
up by microphone 25, and the voice wave signal is taken into 
speech-analyzing/characteristics -extracting means 221A, which 
then analyzes the voice wave, and extracts the characteristic 
of the operation instruction and the characteristic parameter 
of the speech of operator 24. Comparing means 221C compares 
the time series of the resulting characteristic parameter, which 
has been obtained from speech-analyzing/characteristics - 
extracting means 22 lA, with the standard patterns registered 
in storage means 221B, and calculates the similarity between 
them. The operation instruction, "go right , " which corresponds 
to the standard pattern yielding the highest similarity is then 
output from operation instruction selecting means 22 ID to 
operation command converting means 222. 
[0016] 

Operation command converting means 222 converts the 
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operation instruction "go right" into an operation command 
"0001, " which is then transmitted to radio communication device 
23 via communication interface 26. Radio communication device 
23 modulates the received operation command, and sends out the. 
modulated command to construction machine 10 located at 
construction site A. Upon receipt of such an operation coiiunand 
signal received from center B, radio communication device 12 
of construction machine 10 demodulates the operation command 
signal, and then puts the demodulated operation command into 
control circuit 14 via communication interface 13. Control 
circuit 14 converts the received operation command "0001" into 
a control signal for controlling drive unit 101, a hydraulic 
circuitry, of construction machine 10 in such a way that 
construction machine 10 moves to the right, following the speech 
instruction, "go right, " made by operator 24. In this manner, 
it is possible for operator 24 to remotely control construction 
machine 10 with speech instructions, which are input through 
microphone 25, while viewing the monitor screen of monitor 21. 
[0017] 

In the meantime, when remotely controlling TV camera 11 
with speech instructions, as the case of construction machine 
10, operator 24 previously registers standard patterns- While 
viewing the monitor screen of monitor 21, operator 24 utters 
an operation instruction, for example, "zoom in. " The uttered 
operation instruction is taken by microphone 25, and the voice 
wave signal is taken into speech-analyzing/characteristics - 
extracting means 22 lA, which then analyzes the voice wave, and 
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extracts the characteristic of the operation instruction and 
the characteristic parameter of the voice of operator 24. 
Comparing means 221C compares the time series of the resulting 
characteristic parameter, which has been obtained from 
speech-analyzing/characteristics-extracting means 221A, with 
the standard patterns registered in storage means 221B, and 
calculates the similarity between them. The operation 
instruction, ''zoom in, " which corresponds to the standardpat tern 
yielding the highest similarity is then output from operation 
instruction selecting means 221D to operation command converting 
means 222. 
[0018] 

Operation command converting means 222 converts the 
operation instruction "zoom in" into an operation command "1001, " 
which is then transmitted to radio communication device 23 via 
communication interface 26. Radio communication device 23 
modulates the received operation command, and sends out the 
modulated command to construction machine 10 located at 
construction site A. Upon receipt of such an operation command 
signal received from center S, radio communication device 12 
of construction machine 10 demodulates the operation command 
signal, and then puts the demodulated operation command into 
control circuit 14 via communication interface 13. Control 
circuit 14 converts the received operation command "1001" into 
a control signal to control drive unit 111 , the motor for zooming 
in/out, of construction machine 10, in such a way that TV camera 
11 zooms in, following the speech instruction, "zoom in," made 
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by operator 24. In this manner. It is possible for operator 
24 to remotely control TV camera 11 with speech instructions, 
which is input through microphone 25, while viewing the monitor 
screen of monitor 21. 
[0019] 

Under the condition that construction machine 10 and TV 
camera 11 are remotely controlled with speech instructions, it 
might sometimes be difficult to control both constructionmachine 

10 and TV camera 11 at the same time only by using speech 
instructions. For example, when giving the instructions "go 
right" and "zoom in" to construction machine 10 and TV camera 
11, respectively, at the same time, operator 24 can use backup 
remote control unit 201 for manually controlling TV camera 11 
to zoom in. 

[00201 

With the forgoing embodiment of the present invention, 
speech instruction recognizing means 221 recognizes a speech 
instruction made by operator 24. Operation command converting 
means 222 converts the recognition result into an operation 
commandf or constructionmachine 10 or TV camera 11 , and transmits 
the operation command to construction machine 10 located at 
construction site^, so that constructionmachine 10 and TV camera 

11 can be remotely controlled. With this structure, one single 
operator 24 can remotely control construction machine 10 and 
TV camera 11, thereby resulting in labor savings. In addition, 
it is possible to improve the speed and the efficiency of the 
remote controlling of construction machine 10 and TV camera 11. 
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Further, even when it is difficult to operate both construction 
machine 10 and TV camera 11 at the same time, operator 24 is 
able to use backup remote control units 201, 202 mounted on 
construction machine 10 and TV camera 11, respectively, to 
execute manual operation of construction machine 10 or TV camera 
11. 

[0021] 

In the forgoing embodiment, what were remotely controlled 
were construction machine 10 and TV camera 11. The present 
invention should by no means be limited to this, and is also 
applicable to other types of equipment. Likewise, radio 
communication device (second radio communication means) 12 and 
radio communication device (first radio communication means) 
23 were used to transmit and receive operation commands and image 
signals in the foregoing embodiment. The present invention 
should by no means be limited to this , and there could be provided 
two separate communication systems: one is dedicated to 
transmitting and receiving operation commands, and the other 
is dedicated to transmitting and receiving image signals . 
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(57) Abstract: 

PROBLEM TO BE SOLVED: To save labor and also 
operate remotely operated devices remotely and speedily 
with high precision by enabling even one operator to 
remotely operate plural remotely operated devices. 

SOLUTION: The system is equipped with a voice 
Instruction recognizing means 221 which recognizes a 
voice instruction by analyzing the voice wave signal 
outputted from a microphone 25 and an operation 
command converting means 222 which converts the 
recognition result of the voice instruction obtained by 
the voice Instruction recognizing means 221 to an 
operation command to a construction machine 10 and a 
TV camera 11 and the operation command outputted 
from this operation command converting means 222 is 
sent to the construction machine 10 from a radio 
communication device 23. The construction machine 10 
converts the operation command received by a radio 
communication device 12 into a control signal by a 
control circuit 14 and remotely controls the 
construction machine 10 and TV camera 11 according to 
the operation command. 
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VHt=^L, jl«^y^'7i-X2 6?::rt-LTilSm 
fl«l2 3tC3Mill-rS. fc^fHg2 3-C'(±, mf^3-r:^ 

vi^mLximmmAcr)mm.m&i oi,z{^imm-r 
5. ocoMajift«i 2*<-fey:J'-B*»(i>fi] 

ffllHSS 1 4 tcA:>3 . Mm^S^ 1 4 TJi. ilf^ n"? 
>-h' ("OOO 1 J oco 

igiftgB 1 0 1 X'h^mm^^^mm-t^ ztizx'o, :t 
^u-:^2 4imp^hm^^^ ^^izm^j izmtx 
^mmio^^yj\snz^ff^-th. kit, mm^zb 
r. ^-rj- 2 1 <7)m^mmi:M.^*<t^^^ ^ 2 s^ci^ift 

X^-p-^^^^^hZ t J: 0 , MmiSM 1 0 2r#^ 
fc'tfC-SPiS$iJ1l9'f C i: **-C# * . 

[ 0 0 1 7 ] ^ TV7>^ 5 1 1 ■^mpxmmz-m^ 

®S:^^*<^>, TV;*;<7l 1 tC^tTSSfP^^li, 0^1 
Ji-^-f ^2 5tcJ:0e y^r-/rsn. 

tt^^^^w^p8a4aj*a2 2 1 At=jROiis^T.s. 

^^ta«ai#g2 2 1 Ara^^a^sr^WL, Jif^^ 

^iSfc Lr«0#lS^3i-'^U-:? 2 40fflAttSr«i>t-^ 
^S2 2 1 A*»4.f#<i>fUt:itfiayN-7^-:?<0i^^?iJ$rie 



is*S2 2 1 B\z^\^fdmj-^^-ytYcm.mi7. 2 

1 C T'itIS LiliEljS^ tfS:-r & . LT, ib«¥S 2 2 

^mm.^WLl 2 1 D*>f>«if^rjv:^ K^*S2 2 2 
[00 18] jt-f^3-7^H^#^S22 2-C<4. rx- 

j>rr-yTi h^^om^^-^m^ riooij tv^^^f^ 

n-^VHtc^L, afi-f >^7i-x2 6t::/M,ril 
2 3 ti*ifJ-r S . illSfflfiti 2 3 ■C-«±, iKf^n 
K S-SSilil t-Cf^J^JlAcojliSSHslt 1 0 tc[6]{tjll« 
•rS. a^lgaSl O^iSlSBi^illll 2*«-t:y:?-BA>ii> 

:^Hft-f-<ifgPS^i., a^l-f >'^7i-Xl 3S:aLT 
fflfilHS&l 4tA:'3$ti.S. *JW0»1 4t:'(i, 
V^'F r 1 0 0 1 J S:«iJiiPft-f-(C'^t. TV;<?^7 1 

Ycrmm^i 1 iT-s>.ax-Affl^-^5-$ij^ai-rsc:fc 

ICJ:"?, :t^U-:5'2 43&i|6^-rSW?&^^ rx-AT 
•yTj {CjCt.TTV;i^^7l 1 ^-X-AT-y Tiftf^$-<i- 
i.. |5l:m{c:Lr, ^:-:?'2 lC0«7FBffi5:J.^*»' 

/J>v^ ^25 (crfiift-C^^^^S-^-rsc: i: l:: J: 0 , 
TV;&^7 1 \ -k=^-ptc.\^XTmvmth:i'tff^X'% 

c 0 0 1 9 ] mz, #^i^^ic i ^mmmm i o^it/T 

7l 1<0 rx-AT-yrj $:lRlB#{ctf34^«i. — :^r. 
M;itfTV;!>>7l 1<7) rx-AT-yrj SrVN-.y^'T-y 
TfflcoaPi^f^SP2 0 1 2 4*<^f^-rSC: 

[0020] ZffiX 0 =5r*5l*fiOJ^.^ic J:ix«f, mm 
mi ORX/TV:^j(^ 1 lc7)MmcBgLt:5f'<.U-:$'2 
4 A^W-r S ^ W^ffe^fg^S 2 2 1 -CtSMS 

C:coi2ig^mSr^f^3-7>h-3gM#S2 2 2lcJ;'9 
^mmi OJS(.X/TVA:>(yl 1 e^f^^vv Htc^ 
LT, f^31:^Ac7)jaigl^ti!ii OtCjMflt. Z<7)m-i^^ 
•7>-K{cjCtTaiS«itSl 0^tl>'TV;!?>7 1 1 SriSRS 
$iJffllT-# S J: a t= LAcOT-, Jt^«ftS 1 omfTYA^ 
7 1 1 5: 1 A<^:t'^U-^-C'SPi^f^-r?. ^ h-b^X'^ , 
«A:>j-(l:*''^rtg{c=S:S 1 1 tic, a^SM i oao'TV 
;*p< ^ 1 1 tfO^Sf^ S:ffl2I^C*--:>^Stg*(^^f oZtii^ 
T#S. ^Ac. «^«ati!ll'0&t;fTV;<?>7 1 ItcitL 
T, /N'-y;5'T yrfflO#i!l^j®PiMf^Se2 0 1,202 
$rigtt.i.C:i:tCj: 0, SSS^Ml 0S.t/TV;<?^7 1 1 

[ 0 0 2 1 ] ^:t>, ±iec^ll]5feOJg®-C(i, fp^ii^^lc 

J: hmmm^^^mwimm 1 02iVTV;)&^7 1 1 1= 
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1 ORX/TV^:>( y 1 1 Ui-hcoa^s^cov^T tISI 

2iiiaa(i#g) 1 2 timmmm 1 SEjaaft^^ 

[0022] 

izj:ixi£, jmwim^TV:fj^^mcom^(ommzmL 

ZbijiX'^. ^AMtiJ^miz^i, ttiiiz. 

m^c^mmtm^mmizt^-om^mizn ozt *>'t # 
?>ztizxKi, mmiz'^^^^xmmmf^x' # ^ 



[01 ] ^^m<7i^m<mmf.zi5\'fhmmmiyx'rJ^ 

B -fe>':?— 
10 

11 T^//j^^m^) 

12 istMmm (w.2wmm^m 

1 4 mmsi ($ii3P#gg) 

1 5 Mfiijaa^gg 

2 1 ^-r? 

20 1, 202 ^N'-y^r-yrfflilH«f^a5 

2 3 tei«fflfi« (HI «^ia<i#s) 

2 4 

10 1, 111 fgisss 

2 2 1 

222 Jif^=7vyH^#g 

22 1 A w^tfi-^pmattas^e 

22 IB leit^g 

2 2 1 C J:b«?¥g 

2 2 1 D mp^^mmi^^ 



[02] 
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F^^-A(##) 2D003 AAOl AA02 BA04 BA06 CA06 

DA04 FA02 
5H301 AA03 AAIO BB02 BB03 CC03 

CC06 CCIO DD06 DD17 GG09 

KK08 KKIO KK18 KK19 
5K048 AA05 AA08 BAIO BA21 DAOl 

DB05 EB02 EB14 FB05 HA04 

HA06 



